1. Technical Field
The present invention relates generally to scaling of encoded video, and more particularly relates to a system and method for selecting a frequency weighting (FW) matrix for a system implementing Fine-Granularity-Scalability (FGS) technology.
2. Related Art
The Fine-Granularity-Scalability (FGS) coding profile was adopted as part of the MPEG-4 standard in March 2001. The MPEG-4 FGS profile encodes a video sequence into two bit streams with different transmission priorities that can accommodate a large range of bit-rates: the base layer (BL) video stream and the enhancement layer (EL) video stream. The BL is coded using the MPEG-4 non-scalable coding scheme that employs motion-compensation and block-based DCT (discrete cosine transform) coding. The BL is coded to an acceptable minimal bit-rate (the base-layer bit-rate), such that the available bandwidth over the time-varying network is higher than the base-layer bit-rate. The EL codes the difference between the original and the BL signals in the DCT-domain using bit-plane coding.
At the enhancement layer encoder side, these DCT-residual bit-planes are compressed in a progressive (fine-granular) manner, from the most significant bit-plane (MSB) to the least significant bit-plane (LSB). Then, at transmission time, depending on the bandwidth available through the network or decoder capability, only part of the EL may be transmitted. FGS technology is especially useful for video streaming over networks with varying bandwidth, such as Internet video streaming, Internet broadcasting, wireless video communication for both cellular and in-home networks, etc.
FGS consists of a rich set of video coding tools that support various scalability structures and enhance the output visual quality. Frequency weighting (FW) is one such tool that is especially useful for improving visual quality for low bit-rate coding. For example, it is commonly known that the base layer DCT coefficients generally distribute their energy along the zigzag scan line from the top left to the bottom right of the DCT block. Accordingly, the enhancement layer DCT residual blocks inherit a similar zigzag energy distribution pattern. Hence, to ensure good coding quality for lower bandwidth restrictions, the higher energy residuals need to be transmitted in a prioritized manner. The FW method allows bit-plane shifting of selected EL DCT residuals. Therefore, a “frequency weighting” matrix, Mfw, of the same size as the DCT residual block is defined where each element Mfw(i) of the matrix indicates the number of bitplanes that the ith DCT-coefficient should be shifted by.
FIG. 3 illustrates the benefit of FW at low bit-rates. On the left-hand side, the DCT residuals (depicted as vertical lines) of an EL block are shown for the case FW is not used and on the right-hand side, the DCT residuals of an EL block are shown for the case FW is used. As can be seen, each EL block includes several bit-planes, with the MSB located at the top. Within the planes, DCT coefficient residuals extend upward toward the MSB. In the left-hand case, at low bit-rates, if all the bit-planes below the MSB are truncated at the server, the decoder will not receive the DCT coefficient residuals in the first quadrant of the EL block. For most video sequences, the lower accuracy of the DC and first AC's EL residuals translates in a reduced visual quality at the decoder side. Alternatively, if an FW matrix is used where the first quadrant in the DCT block has Mfw(i)=2, as shown in the right-hand side, the DC and first AC's EL residuals will be successfully coded into the MSB, thereby guaranteeing their (at least partial) transmission even at low bit-rates.
Similar to other video coding standards, MPEG-4 standardizes only the FW syntax and its associated semantic meaning for the decoder. Hence, it is the task of the system designer to define innovative algorithms that use the FW syntax in such a manner that the visual quality of the FGS codec can be considerably improved. To achieve FW for FGS coding, one of the key steps is the FW matrix selection. One could select a generic FW matrix based on the zigzag energy distribution characteristics by giving the lower frequency coefficients higher weights and vice versus. However, the generic energy dissipation guideline cannot provide hints for determining the exact quantitative values of the FW matrix. Accordingly, a need exists for effectively selecting an FW matrix.